Information processing method and information processing system

ABSTRACT

An information processing method includes: obtaining, from vehicles, sensor data obtained from sensors of the vehicles; performing object detection processes each using a different one of the sensor data; determining a style of a presentation for monitoring or controlling the vehicles, using results of the object detection processes; and controlling the presentation made by a presentation device according to the style determined.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.16/507,575 filed Jul. 10, 2019, which claims the benefit of priority ofJapanese Patent Application Number 2018-136099 filed on Jul. 19, 2018,and the benefit of priority of Japanese Patent Application Number2019-047427 filed on Mar. 14, 2019, the entire contents of which arehereby incorporated by reference.

BACKGROUND 1. Technical Field

The present disclosure relates to an information processing method andan information processing system.

2. Description of the Related Art

In recent years, self-driving techniques have been introduced andincreasingly researched and developed. It is expected that, along withthe advancement of the self-driving techniques, unmanned self-drivingvehicles will become widely used in the future. From a standpoint ofsafety, the operation of unmanned self-driving vehicles requiresmonitoring of the vehicles by operators in remote places. Sometechniques for monitoring vehicles have been proposed.

For example, Patent Literature (PTL) 1 (Japanese Unexamined PatentApplication Publication No. 2017-147626) discloses a technique fordetermining a remote control device that sends a remote control requestto a control center when a vehicle enters a remote control target area,and allows the control center to remotely control the vehicle for whichthe remote control request has been sent.

Moreover, PTL 2 (Japanese Unexamined Patent Application Publication No.2015-182623) discloses a technique for changing the departure times oftrains according to the number of passengers entering a station.

SUMMARY

In the conventional technique disclosed in PTL 1, when vehicles enter aremote control target area, an operator simultaneously considers thevehicles as remote control targets. As a result, a single operator findsit difficult to remotely control the vehicles at the same time.

Moreover, the conventional technique disclosed in PTL 2 is intended foran operator in a single vehicle, and does not assume that a singleoperator manages vehicles at the same time.

In view of the above, the present disclosure provides an informationprocessing method, an information processing apparatus, and aninformation processing system that make it possible to reduce the burdenof an operator who monitors or controls vehicles while assuring safetyfor the vehicles by the monitoring or controlling.

In order to achieve the above object, an information processing methodaccording to one aspect of the present disclosure includes: obtaining,from vehicles, sensor data obtained from sensors of the vehicles;performing object detection processes each using a different one of thesensor data; determining a style of a presentation for monitoring orcontrolling the vehicles, using results of the object detectionprocesses; and controlling the presentation made by a presentationdevice according to the style determined.

It should be noted that these general and specific aspects may beimplemented using a system, a method, an integrated circuit, a computerprogram, or a computer-readable recording medium such as a CD-ROM, orany combination of systems, methods, integrated circuits, computerprograms, or recording media.

The information processing method etc. of the present disclosure make itpossible to reduce the burden of an operator who monitors or controlsvehicles while assuring safety for the vehicles by the monitoring orcontrolling.

BRIEF DESCRIPTION OF DRAWINGS

These and other objects, advantages and features of the disclosure willbecome apparent from the following description thereof taken inconjunction with the accompanying drawings that illustrate a specificembodiment of the present disclosure.

FIG. 1 is a block diagram of an information processing system accordingto Embodiment 1.

FIG. 2A is a table illustrating priority level information according toEmbodiment 1.

FIG. 2B is a table illustrating a database according to Embodiment 1.

FIG. 2C is a table illustrating a database according to Embodiment 1.

FIG. 3 is a sequence diagram illustrating the operation of theinformation processing system according to Embodiment 1.

FIG. 4A is a diagram illustrating a presentation made by a terminalaccording to Embodiment 1.

FIG. 4B is a diagram illustrating a presentation made by the terminalaccording to Embodiment 1.

FIG. 4C is a diagram illustrating a presentation made by the terminalaccording to Embodiment 1.

FIG. 4D is a diagram illustrating a presentation made by the terminalaccording to Embodiment 1.

FIG. 4E is a diagram illustrating a presentation made by the terminalaccording to Embodiment 1.

FIG. 5 is a flow chart illustrating determination of a priority level bythe operation of an information processing apparatus according toEmbodiment 1.

FIG. 6 is a flow chart illustrating calculation of a boarding andalighting completion time using a database by the operation of theinformation processing apparatus according to Embodiment 1.

FIG. 7 is a block diagram of an information processing system accordingto Embodiment 2.

FIG. 8A is a table illustrating a database according to Embodiment 2.

FIG. 8B is a table illustrating a database according to Embodiment 2.

FIG. 8C is a table illustrating a database according to Embodiment 2.

FIG. 9 is a sequence diagram illustrating the operation of theinformation processing system according to Embodiment 2.

FIG. 10A is a diagram illustrating a presentation made by a terminalaccording to Embodiment 2.

FIG. 10B is a diagram illustrating a presentation made by the terminalaccording to Embodiment 2.

FIG. 10C is a diagram illustrating a presentation made by the terminalaccording to Embodiment 2.

FIG. 10D is a diagram illustrating a presentation made by the terminalaccording to Embodiment 2.

FIG. 10E is a diagram illustrating a presentation made by the terminalaccording to Embodiment 2.

FIG. 10F is a diagram illustrating a presentation made by the terminalaccording to Embodiment 2.

FIG. 11 is a flow chart illustrating determination of a priority levelby the operation of an information processing apparatus according toEmbodiment 2.

DETAILED DESCRIPTION OF THE EMBODIMENTS

An information processing method according to one aspect of the presentdisclosure includes: obtaining, from vehicles, sensor data obtained fromsensors of the vehicles; performing object detection processes eachusing a different one of the sensor data; determining a style of apresentation for monitoring or controlling the vehicles, using resultsof the object detection processes; and controlling the presentation madeby a presentation device according to the style determined.

Since a presentation for monitoring or controlling (hereinaftercollectively and simply also referred to as monitoring) vehicles is madein a style determined using the results of objection detectionprocesses, an operator who monitors vehicles can easily find outvehicles to be monitored. In other words, it is possible to reduce thenumber of vehicles to be simultaneously monitored by the operator whilediminishing the operator's risk for overlooking vehicles to bemonitored. Accordingly, it is possible to reduce the burden of theoperator who monitors vehicles while assuring safety for vehicles by themonitoring.

An information processing apparatus according to one aspect of thepresent disclosure includes: an obtaining unit that obtains, fromvehicles, sensor data obtained from sensors of the vehicles; a detectionprocessor that performs object detection processes each using adifferent one of the sensor data; a determination unit that determines astyle of a presentation for monitoring or controlling the vehicles,using results of the object detection processes; and a presentationcontroller that controls the presentation made by a presentation deviceaccording to the style determined.

An information processing system according to one aspect of the presentdisclosure includes the information processing apparatus and thepresentation device.

The information processing apparatus and the information processingsystem produce the same advantageous effects as above.

The information processing method according to one aspect of the presentdisclosure further includes: determining priority levels of the vehiclesbased on the results of the object detection processes; and determiningthe style of the presentation according to the priority levels.

Since a presentation device makes a presentation for monitoring orcontrolling vehicles according to priority levels of the vehicles, anoperator who monitors the vehicles can find out vehicles to bepreferentially monitored among the vehicles. For this reason, theoperator can monitor any vehicle having a higher priority level thanthose of other vehicles, and thus it is possible to reduce the burden ofthe operator while assuring safety for vehicles.

Moreover, the information processing method according to one aspect ofthe present disclosure further includes: obtaining states of thevehicles from the vehicles; and determining the priority levels based onthe results of the object detection processes and the states of thevehicles.

It is possible to determine priority levels of vehicles moreappropriately using not only the results of object detection processesbut also the states of the vehicles. This allows an operator whomonitors vehicles to further easily find out vehicles to bepreferentially monitored among the vehicles. For this reason, theoperator can further easily monitor any vehicle having a higher prioritylevel than those of other vehicles, and thus it is possible to furtherreduce the burden of the operator while assuring safety for vehicles.

Moreover, in the information processing method according to one aspectof the present disclosure, in each of the object detection processes, atleast one of a boarding passenger and an alighting passenger is detectedfor a stopped vehicle, and the style of the presentation is determinedaccording to the at least one of the boarding passenger and thealighting passenger detected.

It is conventionally difficult to control timing for departure bydetermining completion of boarding a vehicle or alighting from avehicle. For example, it is difficult to determine whether a personclose to a vehicle is a person planning to board, based on sensor data.When an erroneous determination is made, a boarding and alightingpassenger may be put in danger. On the other hand, although it isconceivable that a monitoring operator controls timing for departure,constant monitoring of boarding and alighting states of vehiclesincreases the burden of the operator. In view of the above, with thepresent configuration, since a style of a presentation for monitoring iscontrolled according to boarding and alighting passengers, an operatorcan easily find out vehicles to be monitored. Accordingly, it ispossible to reduce the burden of the operator while assuring safety forvehicles at the time of boarding and alighting. It should be noted thatthe present configuration is effective even when a departure isautomatically controlled. This is because the departure may cause dangeras described above, and the operator is required to preferentiallyperform monitoring.

Moreover, in the information processing method according to one aspectof the present disclosure, in each of the object detection processes, anattribute or a count of at least one of the boarding passenger and thealighting passenger is detected, and the style of the presentation isdetermined according to the attribute or the count detected.

Since a presentation is made in a style according to elements regardingpriority levels for monitoring at the time of boarding and alighting, anoperator can more accurately find out vehicles to be monitored.

Moreover, the information processing method according to one aspect ofthe present disclosure further includes: calculating a time forcompletion of at least one of the boarding passenger boarding thestopped vehicle and the alighting passenger alighting from the stoppedvehicle, using the attribute or the count; and determining the style ofthe presentation according to the time calculated.

Since a presentation is made in a style according to boarding andalighting time, it is possible to present vehicles to an operatoraccording to timing for monitoring. Accordingly, it is possible topresent vehicles to be monitored in a more timely manner. For example,it is possible to present, as a vehicle to be preferentially monitored,a vehicle having a calculated time shorter than that of another vehicle.

Moreover, in the information processing method according to one aspectof the present disclosure, in each of the object detection processes, anobject outside a moving vehicle and a position of the object aredetected, and the style of the presentation is determined according tothe object and the position of the object detected.

Conventionally, there is a technique for controlling the operation of avehicle according to a state of an object outside the vehicle. However,the state of the object varies, and it is difficult to perform controlassuming everything. On the other hand, it is conceivable that amonitoring operator checks the outside of a vehicle, constant monitoringof the outsides of vehicles increases the burden of the operator. Inview of the above, with the present configuration, since a style of apresentation for monitoring is controlled according to objects outsidevehicles, an operator can easily find out vehicles to be monitored.Accordingly, it is possible to reduce the burden of the operator whileassuring safety for vehicles when objects are located outside vehicles.

Moreover, the information processing method according to one aspect ofthe present disclosure further includes: calculating, for each of thevehicles, at least one of a difficulty level of self-driving by thevehicle and a danger level outside the vehicle, using the object and theposition of the object; and determining the style of the presentationaccording to the at least one of the difficulty level and the dangerlevel calculated.

Since a presentation is made in a style according to at least one ofdifficulty levels of self-driving and danger levels outside vehicles, anoperator can more accurately find out vehicles to be monitored. Forexample, it is possible to present, as a vehicle to be preferentiallymonitored, a vehicle having at least one of a relatively high difficultylevel of self-driving and a relatively high danger level outside thevehicle.

Moreover, in the information processing method according to one aspectof the present disclosure, the presentation includes displaying an imageshowing at least one of an entrance and an exit of each of the vehicles.

For this reason, an operator can easily check safety at the time ofboarding and alighting.

Moreover, in the information processing method according to one aspectof the present disclosure, the presentation includes displaying an imageshowing an outside of each of the vehicles.

For this reason, an operator can easily check safety outside a vehicle.

Moreover, in the information processing method according to one aspectof the present disclosure, the determining of the style of thepresentation includes providing a higher visibility of the presentationfor a higher priority level that is a priority level included in thepriority levels.

Since a higher visibility is provided for a vehicle having a higherpriority level, an operator can quickly check information regardingvehicles to be monitored.

Moreover, in the information processing method according to one aspectof the present disclosure, the determining of the style of thepresentation includes providing a greater amount of information in thepresentation for a higher priority level that is a priority levelincluded in the priority levels.

Since a greater amount of information is provided for a vehicle having ahigher priority level in a presentation, an operator can understand thedetails of information regarding vehicles to be monitored.

Further, since a less amount of information is provided for a vehiclehaving a lower priority level, it is possible to reduce frame rates ofvehicles to be presented to the operator, and degrade image quality. Forthis reason, it is possible to reduce network load among devices,vehicles, and presentation devices that execute this informationprocessing method.

Moreover, in the information processing method according to one aspectof the present disclosure, the determining of the style of thepresentation includes emphasizing the presentation further for a higherpriority level that is a priority level included in the priority levels.

Since a presentation is emphasized further for a vehicle having a higherpriority level, an operator can quickly identify vehicles to bemonitored.

Moreover, in the information processing method according to one aspectof the present disclosure, the determining of the style of thepresentation includes starting the presentation when a priority levelincluded in the priority levels exceeds a threshold value.

Since only a vehicle having a priority level higher than or equal to athreshold value is presented, an operator can focus on vehicles to bemonitored.

Moreover, in the information processing method according to one aspectof the present disclosure, the determining of the style of thepresentation includes arranging the presentation in order of thepriority levels.

Since vehicles are presented in order of priority levels, an operatorcan quickly identify vehicles to be monitored. In addition, the operatorcan check a vehicle to be monitored next in advance, which can makemonitoring more efficient.

Moreover, in the information processing method according to one aspectof the present disclosure, the priority levels are further determinedbased on at least one of positions of the vehicles, times correspondingto the positions of the vehicles, histories of required times for aboarding passenger to complete boarding and required times for analighting passenger to complete alighting, and weather informationcorresponding to the positions of the vehicles.

With this, it is possible to set priority levels to vehicles moreappropriately. For this reason, it is possible to cause a presentationdevice to present vehicles to be monitored more appropriately.

It should be noted that these general and specific aspects may beimplemented using a system, a method, an integrated circuit, a computerprogram, or a computer-readable recording medium such as a CD-ROM, orany combination of systems, methods, integrated circuits, computerprograms, or recording media.

Hereinafter, embodiments will be described in detail with reference tothe drawings. It should be noted that each of the embodiments describedhereinafter shows a specific example of the present disclosure. Thenumerical values, shapes, materials, structural components, thearrangement and connection of the structural components, steps, theprocessing order of the steps, etc., shown in the following embodimentsare mere examples, and are therefore not intended to limit the presentdisclosure. Furthermore, among the structural components in thefollowing embodiments, structural components not recited in any one ofthe independent claims defining the most generic concept of the presentdisclosure are described as optional structural components.

It should be noted that the figures are schematic diagrams and are notnecessarily precise illustrations. Furthermore, in the figures,substantially identical components are assigned the same referencesigns, and overlapping description thereof may be omitted or simplified.

In what follows, an information processing method, an informationprocessing apparatus, and an information processing system according toeach of the embodiments of the present disclosure will be described.

EMBODIMENT 1

(Configuration)

FIG. 1 is a block diagram of information processing system 1 accordingto Embodiment 1.

Information processing system 1 includes vehicle 3, informationprocessing apparatus 5, and terminal 7.

Vehicle 3 is a self-driving vehicle that can be driven by remote controlby a remote control operator thereof, and can self-drive when notremotely controlled. Examples of vehicle 3 include a bus, a tax, and atrain. The present embodiment assumes that vehicle 3 is a bus and hasstopped at a stop (bus stop). Hereinafter, a person who gets on or plansto get on vehicle 3 is referred to as a boarding passenger, and a personwho gets off or plans to get off vehicle 3 is referred to as analighting passenger. A term indicating both boarding passengers andalighting passengers is referred to as boarding and alightingpassengers.

Vehicle 3 includes first obtaining unit (first obtainer) 31, secondobtaining unit (second obtainer) 32, third obtaining unit (thirdobtainer) 33, fourth obtaining unit (fourth obtainer) 34, andcommunication unit (communicator) 35.

First obtaining unit 31 is a device that obtains information indicatingboarding and alighting start times, and is, for example, a clock.Specifically, a boarding and alighting start time indicates a time atwhich a boarding and alighting passenger starts boarding or alightingfrom vehicle 3 when vehicle 3 has stopped at a stop. First obtainingunit 31 transmits the information indicating the boarding and alightingstart times to information processing apparatus 5. It should be notedthat a boarding and alighting start time may be a time at which vehicle3 is stopped at a stop by automated driving.

Second obtaining unit 32 obtains vehicle position information (e.g.global positioning system (GPS) information) indicating the currentposition of vehicle 3. Second obtaining unit 32 transmits the obtainedvehicle position information to information processing apparatus 5 andterminal 7. The current position of vehicle 3 is an example of theposition of vehicle 3.

Third obtaining unit 33 is a device that obtains boarding and alightingpassenger images that are captured images of boarding and alightingpassengers boarding and alighting from vehicle 3, and is, for example, acamera. Specifically, third obtaining unit 33 generates boarding andalighting passenger images by capturing people. (e.g. boardingpassengers) waiting at a stop of vehicle 3 and people (e.g. alightingpassengers) standing up in vehicle 3. Third obtaining unit 33 transmitsthe boarding and alighting passenger images to information processingapparatus 5. Third obtaining unit 33 is an example of a sensor.Moreover, a boarding and alighting passenger image is an example ofsensor data.

Fourth obtaining unit 34 is a device that obtains vehicle inside andoutside images each showing the inside and outside of vehicle 3, and is,for example, a camera. A vehicle inside and outside image includesvehicle's surroundings, that is, information about pedestrians,vehicles, bicycles, etc. outside vehicle 3, information shout passengersinside vehicle 3, and information about boarding and alightingpassengers close to the entrance and exit of vehicle 3. Fourth obtainingunit 34 transmits the vehicle inside and outside images to terminal 7.It should be noted that fourth obtaining unit 34 may be used for thirdobtaining unit 33, and boarding and alighting passenger images may bevehicle inside and outside images.

Communication unit 35 is a communication interface that transmitsinformation indicating boarding and alighting start times, vehicleposition information, and boarding and alighting passenger images toinformation processing apparatus 5, and transmits the vehicle positioninformation and vehicle inside and outside images to terminal 7. Inaddition, communication unit 35 receives control commands for remotecontrol when vehicle 3 is remotely controlled by a remote controloperator. Communication unit 35 outputs control commands to an actuator,an indicator, etc. (not shown) that control vehicle 3.

Here, there are vehicles 3, and each of vehicles 3 has a vehicle ID foridentifying itself. Communication unit 35 transmits informationindicating boarding and alighting start times, vehicle positioninformation, and boarding and alighting passenger images associated withthe respective vehicle IDs to information processing apparatus 5, andtransmits vehicle inside and outside images associated with therespective vehicle IDs to terminal 7.

Information processing apparatus 5 is a device capable of remotelycontrolling vehicles 3 that are self-driving, according to aninstruction from an operator in a remote place. Information processingapparatus 5 includes detection processor 51, determination unit(determiner) 52, storage controller 53, communication unit 54, andpresentation controller 55.

Detection processor 51 performs object detection processes. Examples ofan object detection process include detecting at least one of boardingpassengers and alighting passengers for vehicle 3 that has stopped, anddetecting at least one of boarding passenger attributes or count andalighting passenger attributes or count. Specifically, detectionprocessor 51 obtains boarding and alighting passenger images from thirdobtaining unit 33 of each vehicle 3, detects a boarding and alightingpassenger count, boarding and alighting passenger attributes, etc.,based on the boarding and alighting passenger images, and generatesboarding and alighting passenger information for each vehicle 3. Aboarding and alighting passenger attribute indicates information such asadult, child, and elderly. Detection processor 51 outputs boarding andalighting passenger information to determination unit 52.

Determination unit 52 determines a style of a presentation formonitoring or controlling vehicles 3, using the results of the objectdetection processes. Styles of presentation will be described in detaillater.

Moreover, determination unit 52 obtains boarding and alighting passengerinformation from detection processor 51, and obtains previous boardingand alighting data from a database stored in storage controller 53 whendetermination unit 52 obtains a boarding and alighting start time andvehicle position information from first obtaining unit 31 and secondobtaining unit 32 of each vehicle 3. Specifically, determination unit 52requests, from storage controller 53, previous boarding and alightingdata similar to the boarding and alighting passenger information, theboarding and alighting start time, and the vehicle position information,and obtains the similar previous boarding and alighting data fromstorage controller 53.

Determination unit 52 obtains previous boarding and alighting data fromstorage controller 53, and determines, for respective vehicles 3,priority levels that are indicators for monitoring vehicles 3 on apriority basis. As shown in FIG. 2A, determination unit 52 transmitspriority level information in which the determined priority levels areassociated with the vehicle IDs of vehicles 3 to terminal 7. FIG. 2A isa table illustrating priority level information according to Embodiment1.

Storage controller 53 stores previous boarding and alighting data ofwhich a database is made. Boarding and alighting data is informationsuch as boarding and alighting start times, stops, boarding andalighting passenger attributes, boarding and alighting passenger countseach for a different one of boarding and alighting passenger attributes,required times for all boarding and alighting passengers to completeboarding and alighting, and boarding and alighting times eachcorresponding to a different one of the boarding and alighting passengerattributes. In response to a request from determination unit 52, storagecontroller 53 searches for previous boarding and alighting data similarto boarding and alighting passenger information, and transmits thesimilar previous boarding and alighting data to determination unit 52.It should be noted that storage controller 53 may store a trained modeltrained using previous boarding and alighting data, instead of theprevious boarding and alighting data of which the database is made. Inthis case, storage controller 53 transmits the trained model todetermination unit 52 in response to a request from determination unit52.

FIG. 2B and FIG. 2C show boarding and alighting data. FIG. 2B and FIG.2C each are a table illustrating boarding and alighting data accordingto Embodiment 1. As shown in FIG. 2B, the boarding and alighting datainclude “TIME” when vehicle 3 stopped at a stop, “LOCATION” at whichvehicle 3 stopped, “BOARDING AND ALIGHTING PASSENGER ATTRIBUTE ANDCOUNT,” and “REQUIRED TIME” for all boarding and alighting passengers tocomplete boarding and alighting. As shown in FIG. 2C, the boarding andalighting data further includes “BOARDING AND ALIGHTING TIME” per personin connection with each boarding and alighting passenger attribute.

As shown in FIG. 1, communication unit 54 is a communication interfacethat receives information indicating boarding and alighting start times,vehicle position information, and boarding and alighting passengerimages, and transmits priority level information to terminal 7.Communication unit 54 is an example of an obtaining unit.

Presentation controller 55 controls a style of a presentation made byterminal 7 according to the style of the presentation determined bydetermination unit 52. In other words, presentation controller 55outputs a command for controlling a presentation made by terminal 7 sothat terminal 7 makes the presentation in the style according topriority levels determined.

Terminal 7 is an operating terminal used by an operator to monitor themoving state of vehicle 3 and remotely operate vehicle 3, and is, forexample, a personal computer and a tablet terminal. Terminal 7 includespresentation unit (presenter) 71, operating unit (operator) 72, timer73, and communication unit 74.

Presentation unit 71 is a monitor that presents vehicle inside andoutside images, vehicle position information, etc. for each vehicle 3.Specifically, presentation unit 71 presents, for each vehicle 3,information regarding vehicle 3 such as a vehicle inside image, anentrance and exit image, an image of the outside of vehicle 3 (thesurroundings of vehicle 3), a priority level, a boarding and alightingcompletion time, a vehicle ID and a stop.

Presentation unit 71 presents information regarding vehicles 3 invarious styles. Presentation unit 71 presents, for example, theinformation regarding vehicle 3 having the highest priority level with alarge-area image, only the information regarding vehicle 3 having apriority level higher than or equal to a threshold value among vehicles3, the information regarding vehicle 3 having a high priority level indetail, information regarding vehicles 3 arranged in the order accordingto their priority levels with same-size images, and vehicle 3 having thehighest priority level whose image is emphasized by being colored.

Operating unit 72 is an input unit that receives remote control ofvehicle 3 by an operator. The operator checks whether all boarding andalighting passengers have boarded and alighted (check on completion ofboarding and alighting), and operates operating unit 72 when vehicle 3is caused to depart. Operating unit 72 is, for example, a touch paneland a keyboard. Operating unit 72 transmits control commands for remotecontrol to vehicle 3 via communication unit 74.

Timer 73 is, for example, a clock that measures the time when theoperator operates operating unit 72 to transmit a control command forcausing vehicle 3 to depart.

Communication unit 74 is a communication interface that transmitscontrol commands to vehicle 3.

It should be noted that, in the present embodiment, timer 73 is not anessential component in terminal 7, and second obtaining unit 32 is notan essential component in vehicle 3. For this reason, timer 73 andsecond obtaining unit 32 need not be included, and any one of timer 73and second obtaining unit 32 may be provided.

(Operation)

Next, the following describes the operation of the informationprocessing method, information processing apparatus 5, and informationprocessing system 1 according to the present embodiment. Specifically,the following describes the operation of information processing system 1performed from when vehicle 3 stops at a bus stop to when vehicle 3departs after a certain period of time.

FIG. 3 is a sequence diagram illustrating the operation of informationprocessing system 1 according to Embodiment 1.

As shown in FIG. 3, vehicle 3 obtains information indicating a boardingand alighting start time, and transmits the obtained information toinformation processing apparatus 5 (step S101). In addition, vehicle 3obtains vehicle position information indicating the current position ofvehicle 3, and transmits the obtained vehicle position information toinformation processing apparatus 5 and terminal 7 (step S102).Determination unit 52 of information processing apparatus 5 receives theinformation indicating the boarding and alighting start time and thevehicle position information via communication unit 54.

Moreover, vehicle 3 obtains boarding and alighting passenger images thatare captured images of boarding and alighting passengers boarding andalighting from vehicle 3, and transmits the obtained boarding andalighting passenger images to information processing apparatus 5 (stepS103). Detection processor 51 of information processing apparatus 5receives the boarding and alighting passenger images via communicationunit 54.

In the mean time, vehicle 3 obtains vehicle inside and outside images,and transmits the obtained vehicle inside and outside images to terminal7 (step S104).

Detection processor 51 generates boarding and alighting passengerinformation by detecting a boarding and alighting passenger count,boarding and alighting passenger attributes, etc., based on the boardingand alighting passenger images received in step S103 (step S105).Detection processor 51 outputs the boarding and alighting passengerinformation to determination unit 52 (step S106).

Determination unit 52 requests, from storage controller 53, previousboarding and alighting data similar to the obtained hoarding andalighting passenger information, boarding and alighting start time, andvehicle position information (step S107).

In response to the request from determination unit 52, storagecontroller 53 searches for the previous boarding and alighting datasimilar to the boarding and alighting passenger information, theboarding and alighting start time, and the vehicle position information,and outputs the similar previous boarding and alighting data todetermination unit 52 (step S108). Boarding and alighting data includesinformation such as times at which vehicle 3 stops at stops, stops,boarding and alighting passenger attributes, counts each for a differentone of boarding and alighting passenger attributes, required times fromwhen all boarding and alighting passengers start boarding and alightingto when the all boarding and alighting passengers finish boarding andalighting, boarding and alighting times each for a different one of theboarding and alighting passenger attributes, as shown in FIG. 2B andFIG. 2C. It should be noted that storage controller 53 may transmit atrained model to determination unit 52 in response to a request fromdetermination unit 52, instead of outputting similar previous boardingand alighting data in response to a request from determination unit 52.

Determination unit 52 obtains the previous boarding and alighting datafrom storage controller 53, and sets a priory level for monitoring foreach vehicle 3 (step S109). Determination unit 52 determines a prioritylevel for each vehicle 3 according to the boarding and alightingpassenger information, the boarding and alighting start time, and thevehicle position information. The priority levels will be describedlater. Determination unit 52 transmits, to terminal 7, priority levelinformation in which the set priory levels are associated with thevehicle IDs (step S110). It should be noted that determination unit 52may obtain a trained model from storage controller 53 instead ofobtaining previous boarding and alighting data from storage controller53, and set a priority level for monitoring for each vehicle 3. In thiscase, determination unit 52 determines the priority level for eachvehicle 3 using the obtained trained model.

Here, determination unit 52 determines that information regardingvehicle 3 is to be presented by terminal 7 in a more highly visiblestyle of a presentation as the priority level is higher, a style of apresentation in which an amount of information is greater as thepriority level is higher, or a more emphasized style of a presentationas the priority level is higher. In addition, determination unit 52determines that information regarding vehicles 3 is to be presented in astyle of a presentation in which vehicles 3 are arranged in order ofdecreasing priority level. Presentation controller 55 controls thepresentation made by terminal 7 according to a determined style, thatis, causes terminal 7 to start presenting the information regardingvehicle 3 when the priority level exceeds a threshold value.

Next, terminal 7 receives the vehicle inside and outside image obtainedin step S104 and the priority level information obtained in step S110.Terminal 7 makes a presentation according to the priority levels (stepS111). Terminal 7 makes a presentation in various styles according tothe priority levels. FIG. 4A to FIG. 4E each illustrate a style of apresentation made by presentation unit 71.

FIG. 4A is a diagram illustrating a presentation made by terminal 7according to Embodiment 1. As shown in FIG. 4A, presentation unit 71presents vehicle 3 having the highest priority level with the large-areaimage (in a highly visible style). Specifically, presentation unit 71presents only vehicles 3 having at least a predetermined priority level(threshold value) among vehicles 3, and vehicle 3 having the highestpriority level with a large-area image. The information regardingvehicle 3 having the highest priority level indicates the required timefor all passengers to complete boarding and alighting from vehicle 3,the images regarding vehicle 3, and the stop. The images regardingvehicle 3 include the vehicle inside image, the entrance and exit image,the vehicle outside images, etc. Moreover, for example, presentationunit 71 presents, for vehicles 3 having the second highest prioritylevel or a lower priority level, information simpler than that ofvehicle 3 having the highest priority level. Specifically, presentationunit 71 presents only one kind of images regarding vehicles 3, such asimages whose frame rates are reduced and images whose quality isreduced. Furthermore, presentation unit 71 does not present informationregarding vehicle 3 having a priority level lower than the predeterminedpriority level, that is, vehicle 3 having the sixth highest prioritylevel or a lower priority level in the present embodiment.

FIG. 4B is a diagram illustrating a presentation made by terminal 7according to Embodiment 1. As shown in FIG. 4B, presentation unit 71presents only the information regarding vehicles 3 having at least apredetermined priority level among vehicles 3. In FIG. 4B, presentationunit 71 presents only the information regarding vehicles 3 having asimilar boarding and alighting completion time on a priority basis. Inthe present embodiment, presentation unit 71 presents informationregarding two vehicles 3 but not information regarding other vehicles 3having the third highest priority level or a lower priority level.

FIG. 40 is a diagram illustrating a presentation made by terminal 7according to Embodiment 1. As shown in FIG. 4C, presentation unit 71presents the information regarding vehicle 3 having the highest prioritylevel with the large-area image, and only vehicles 3 having at least apredetermined priority level among vehicles 3. In FIG. 4C, presentationunit 71 presents the images regarding vehicles 3.

FIG. 4D is a diagram illustrating a presentation made by terminal 7according to Embodiment 1. As shown in FIG. 4D, presentation unit 71presents all the images regarding vehicles 3 (in a style of apresentation in which an amount of information is increased) at a sizethat is large enough to allow an operator to view the informationregarding vehicles 3. In FIG. 4D, presentation unit 71 presents theinformation regarding vehicles 3 arranged in order of decreasingpriority level.

FIG. 4E is a diagram illustrating a presentation made by terminal 7according to Embodiment 1. As shown in FIG. 4E, presentation unit 71presents the information regarding vehicle 3 leaving the highestpriority level that is emphasized by being colored.

It should be noted that the presentations in FIG. 4A to FIG. 4E are mereexamples, and the present embodiment is not limited to these. Forexample, a presentation includes displaying images each showing at leastone of the entrance and the exit of a different one of vehicles 3. Inaddition, a presentation includes images each showing the outside of adifferent one of vehicles 3.

Here, we will return to the description of the sequence diagram shown inFIG. 3.

When all the passengers complete boarding and alighting from vehicle 3,terminal 7 receives a departure instruction from the operator (stepS112). For example, the operator checks on the completion of boardingand alighting via terminal 7. Then, when the operator determines thecompletion of boarding and alighting, the operator inputs a departureinstruction into terminal 7. Terminal 7 receives the departureinstruction inputted. It should be noted that, in the checking on thecompletion of boarding and alighting, terminal 7 may determine whetherany passenger is close to the entrance and the exit by imagerecognition.

When terminal 7 receives the departure instruction from the operator,terminal 7 transmits to vehicle 3 a control command for causing vehicle3 to depart (step S113).

When vehicle 3 receives the control command transmitted in step S113,vehicle 3 departs (step S114).

Moreover, when terminal 7 receives the departure instruction in stepS112, timer 73 measures a boarding and alighting completion time inresponse to the reception of the departure instruction, and transmitsinformation indicating the boarding and alighting completion time toinformation processing apparatus 5 (step S115).

Next, determination unit 52 stores into storage controller 53 theinformation indicating the boarding and alighting start time received instep S101, and updates a database in storage controller 53 (step S118).

Moreover, determination unit 52 calculates a required time from theinformation indicating the boarding and alighting start time received instep S116 and the information indicating the boarding and alightingcompletion time stored in step S115, stores the required time intostorage controller 53, and updates the database in storage controller 53(step S118). It should be noted that determination unit 52 may update atrained model using boarding and alighting data such as times,locations, boarding and alighting passenger attributes, boarding andalighting passenger counts, and required times, instead of updating theabove database in storage controller 53.

Furthermore, determination unit 52 stores into storage controller 53 theboarding and alighting passenger information obtained in step S106 (stepS117), and updates the database in storage controller 53 (step S118).

In this manner, the processing by information processing system 1 isended. It should be noted that, in information processing system 1, theupdating of the database and trained model in storage controller 53 insteps S115 to S118 is not an essential process, and such a process neednot be performed.

Next, the following describes the operation of information processingapparatus 5 with reference to FIG. 5. Here, processing is performedafter vehicle 3 stops at a predetermined position. FIG. 5 is a flowchart illustrating determination of a priority level by the operation ofinformation processing apparatus 5 according to Embodiment 1.

First, as shown in FIG. 5, detection processor 51 of informationprocessing apparatus 5 receives boarding and alighting passenger imagesfrom each vehicle 3 (step S161). Detection processor 51 generatesboarding and alighting passenger information by detecting a boarding andalighting passenger count, boarding and alighting passenger attributes,etc., based on the boarding and alighting passenger images. Detectionprocessor 51 outputs the boarding and alighting passenger information todetermination unit 52.

Determination unit 52 of information processing apparatus 5 receivesfrom vehicle 3 information indicating a boarding and alighting starttime, and vehicle position information (step S162).

Determination unit 52 determines whether any boarding and alightingpassenger is in vehicle 3 based on the boarding and alight passengerinformation (step S163). Specifically, determination unit 52 determineswhether the boarding and alighting passenger count included in theboarding and alighting passenger information is at least 1.

When the boarding and alighting passenger information indicates one ormore boarding and alighting passengers (YES in step S163), determinationunit 52 calculates a boarding and alighting completion time.Specifically, determination unit 52 calculates a required time from whenall passengers start boarding and alighting to when they completeboarding and alighting, based on the boarding and alighting passengerinformation, previous boarding and alighting data, etc.

Determination unit 52 adds the calculated required time to the boardingand alighting start time obtained in step S162 to calculate the boardingand alighting completion time for each vehicle 3 (step S164).

Next, determination unit 52 sets higher priority levels to vehicles 3 inorder of closeness of the boarding and alighting completion times to thecurrent time. Determination unit 52 sets a priority level to each ofvehicles 3 every time vehicle 3 stops at a stop, and updates thepriority levels of vehicles 3 in order of boarding and alightingcompletion time (step S165). The current time is an example of a time.

In contrast, when the boarding and alighting passenger informationindicates no boarding and alighting passenger (NO in step S163),determination unit 52 sets the highest priority level to vehicle 3determined to include no boarding and alighting passenger (step S167).It should be noted that when there are vehicles 3 including no boardingand alighting passenger, determination unit 52 sets all such vehicles 3as vehicles 3 having the highest priority level. Determination unit 52calculates a boarding and alighting completion time for each of othervehicles 3 in the same manner as step S164, sets priority levels toother vehicles 3, and updates the priority levels of other vehicles 3(step S168).

Next, determination unit 52 transmits information indicating thepriority levels of vehicles 3 to terminal 7 via communication unit 54(step S166). In this manner, the processing by information processingapparatus 5 is ended. The same processing is performed at apredetermined position that is the next stop.

In what follows, step S164 of calculating a boarding and alightingcompletion time in FIG. 5 will be specifically described with referenceto FIG. 6. FIG. 6 is a flow chart illustrating calculation of a boardingand alighting completion time using a database by the operation ofinformation processing apparatus 5 according to Embodiment 1. It shouldbe noted that a boarding and alighting completion time may be calculatedusing a trained model instead of a database.

First, as shown in FIG. 6, determination unit 52 refers to previousboarding and alighting data stored in storage controller 53 (step S171).Specifically, determination unit 52 requests from storage controller 53previous boarding and alighting data similar to boarding and alightingpassenger information, a boarding and alighting start time, and vehicleposition information, and obtains the requested previous hoarding andalighting data from storage controller 53.

Determination unit 52 determines whether the previous boarding andalighting data similar to the boarding and alighting passengerinformation, the boarding and alighting start time, and the vehicleposition information is in storage controller 53 (step S172). Forexample, similar previous boarding and alighting data is previousboarding and alighting data similar to at least one of boarding andalighting passenger information, a boarding and alighting start time,and vehicle position information. Take a boarding and alighting starttime for example, similar previous boarding and alighting data istemporally close to a boarding and alighting start time, such as withinone hour etc. from the boarding and alighting start time.

When the previous boarding and alighting data similar to the boardingand alighting passenger information is determined to be in storagecontroller 53 (YES in step S172), determination unit 52 selects arequired time (previous required time) for all passengers to completehoarding and alighting as the current required time for all passengersto complete boarding and alighting (step S173).

Determination unit 52 calculates a boarding and alighting completiontime by adding the required time selected in step S173 to the boardingstart time (step S174). Subsequently, the processing is finished.

In contrast, when the previous boarding and alighting data similar tothe boarding and alighting passenger information is determined not to bein storage controller 53 (NO in step S172), determination unit 52extracts, from previous boarding and alighting data stored in storagecontroller 53, boarding and alighting passenger attributes and boardingand alighting times each corresponding to a different one of theboarding and alighting passenger attributes (step S175).

Determination unit 52 calculates a required tune for all passengers tocomplete boarding and alighting, based on step S175 (step S176).Determination unit 52 proceeds to step S174 and calculates a boardingand alighting completion time. Subsequently, the processing is finished.

As described above, in the information processing method, informationprocessing apparatus 5, and information processing system 1, when anoperator monitors stopped vehicles 3, it is possible to calculate, foreach vehicle 3, a required time for all passengers to complete boardingand alighting, based on a boarding passenger count, an alightingpassenger count, boarding passenger attributes, and alighting passengerattributes. Information processing apparatus 5 sets a higher prioritylevel for vehicle 3 that requires a shorter boarding and alightingcompletion time, based on a boarding and alighting start time and arequired time. Terminal 7 presents, for example, vehicle 3 having a highpriority level with a large-area image, only vehicle 3 having a prioritylevel higher than or equal to a threshold value, information regardingvehicles 3 arranged in the order according to their priority levels,information regarding vehicles 3 in detail, and vehicle 3 having thehighest priority level whose image is emphasized by being colored. Forthis reason, the operator can monitor only vehicles 3 presented byterminal 7, that is, easily find out the vehicles to be monitored.Accordingly, since the number of vehicles 3 to be simultaneouslymonitored by the operator is reduced, it is possible to diminish theoperator's risk for overlooking vehicles 3 to be monitored. As a result,it is possible to reduce the burden of the operator who monitorsvehicles 3 while assuring safety for vehicles 3 by the monitoring.

EMBODIMENT 2

(Configuration)

In the present embodiment, a case is described in which vehicle 130 isassumed to be self-driving.

Unless otherwise specifically stated, the configurations of aninformation processing method, information processing apparatus 150, andinformation processing system 101 according to the present embodimentare the same as Embodiment 1. The same reference signs are given to thesame components, and the detailed description of the components isomitted.

FIG. 7 is a block diagram of information processing system 101 accordingto Embodiment 2.

As shown in FIG. 7, information processing system 101 includes vehicle130, information processing apparatus 150, and terminal 170.

Vehicle 130 includes fifth obtaining unit (fifth obtainer) 135, sixthobtaining unit (sixth obtainer) 136, seventh obtaining unit (seventhobtainer) 137, and communication unit 138.

Fifth obtaining unit 135 is a device that obtains vehicle stateinformation indicating the state of vehicle 130, and is, for example, anelectronic control unit (ECU). Vehicle state information includesinformation indicating the state of vehicle 130, such as moving,temporary stop, emergency stop, and speed. Fifth obtaining unit 135transmits vehicle state information to information processing apparatus150 and terminal 170.

Sixth obtaining unit 136 obtains vehicle position information (e.g. GPSinformation) indicating the current position of vehicle 130. Sixthobtaining unit 136 transmits the obtained vehicle position informationto information processing apparatus 150 and terminal 170.

Seventh obtaining unit 137 performs object detection processes. Theobject detection processes are each performed to detect any objectoutside a moving vehicle, and the position of the object. Seventhobtaining unit 137 obtains sensor information indicating a passengerstate inside vehicle 130, the position of an object outside vehicle 130,etc. Seventh obtaining unit 137 is an image sensor such as a camera thatcaptures images of the outside and inside of vehicle 130, a distancesensor such as a laser imaging detection and ranging (LIDAR), etc.Seventh obtaining unit 137 transmits sensor information to informationprocessing apparatus 150 and terminal 170. Examples of the objectoutside vehicle 130 include a person walking ahead of vehicle 130 and abicycle behind vehicle 130. Seventh obtaining unit 137 is an example ofa sensor.

Communication unit 138 is a communication interface that transmitsvehicle state information, vehicle position information, and sensorinformation to information processing apparatus 150. In addition,communication unit 138 receives control commands for remote control whenvehicle 130 is remotely controlled by a remote control operator.

Detection processor 151 of information processing apparatus 150 obtainssensor information from seventh obtaining unit 137 of vehicle 130, andgenerates vehicle inside and outside information indicating a passengerstate inside vehicle 130, an object outside vehicle 130, etc., based onthe sensor information. Detection processor 151 outputs the vehicleinside and outside information to determination unit 152.

When determination unit 152 of information processing apparatus 150obtains the vehicle inside and outside information from detectionprocessor 151, determination unit 152 obtains previous vehicle datastored in a database in storage controller 153. Specifically,determination unit 152 requests from storage controller 153 previousvehicle data similar to the vehicle inside and outside information, andobtains the requested previous vehicle data from storage controller 153.It should be noted that determination unit 152 may request from storagecontroller 153 a trained model created based on previous vehicle datastored in a database in storage controller 153, instead of the previousvehicle data, and may obtain the trained model from storage controller153.

Determination unit 152 calculates at least one of a difficulty level ofself-driving by each vehicle 130 and a danger level outside vehicle 130,based on sensor information, previous vehicle data, etc. A difficultylevel of self-driving is a parameter indicating the ease of moving byvehicle 130. For example, when vehicle 130 is moving on a narrow road orvehicle 130 is moving in rain, fog, etc., a difficulty level ofself-driving is high. A danger level outside vehicle 130 is a parameterindicating a possibility of an accident in an area in which vehicle 130is moving. For example, an area with danger of rushing out, an areaunder construction, etc. have a high danger level. Hereinafter, thepresent embodiment will be described using mainly a danger level. Itshould be noted that determination unit 152 may calculate at least oneof a difficulty level of self-driving by each vehicle 130 and a dangerlevel outside vehicle 130 using an obtained trained model instead ofsensor information, previous vehicle data, etc.

Storage controller 153 of information processing apparatus 150 is astorage device that stores previous vehicle data. Vehicle data isinformation, such as a label, a location corresponding to the label, arecommended operation corresponding to the label, and a danger levelcorresponding to the label. A label is a descriptive text about alocation, such as an area with danger of rushing out and underconstruction. In response to a request from determination unit 152,storage controller 153 searches for previous vehicle data similar tovehicle inside and outside information, and transmits the similarprevious vehicle data to determination unit 152. It should be noted thata danger level is optionally changed, and varies depending on, forexample, the number of people outside vehicle 130. It should be notedthat storage controller 153 may store a trained model trained based onprevious vehicle data, instead of the previous vehicle data.

FIGS. 8A, 8B, and FIG. 8C each are a table illustrating previous vehicledata according to Embodiment 2. As shown in FIG. 8A, previous vehicledata includes “LOCATION” in which a dangerous situation has occurred,“LABEL” that is a descriptive text about a dangerous situation,“RECOMMENDED OPERATION” corresponding to a label, and “DANGER LEVEL”corresponding to a label. As shown in FIG. 8B, the previous vehicle datafurther includes “STATE” into which vehicle 130 has gone, such asautonomous emergency braking (AEB) activation and temporary stop,“LABEL” that is a descriptive text corresponding to a state,“RECOMMENDED OPERATION” corresponding to a state, and “DANGER LEVEL”corresponding to a state. As shown in FIG. 8C, the previous vehicle datafurther includes “SITUATION” of the outside of vehicle 130 while vehicle130 is moving, such as a pedestrian on a road and a bicycle behindvehicle 130, “LABEL” that is a descriptive text about a situation,“RECOMMENDED OPERATION” corresponding to a situation, and “DANGER LEVEL”corresponding to a situation.

As shown in FIG. 7, presentation unit 171 of terminal 170 receives viacommunication unit 173 information regarding vehicles 130, such as thepriority levels, states, inside images, outside images, vehicle IDs, andcurrent positions of vehicles 130, and presents the received informationfor each vehicle 130.

Presentation unit 171 of terminal 170 presents, for example, vehicle 130having the highest priority level with a large-area image, only vehicle130 having a priority level higher than or equal to a threshold valueamong vehicles 130, information regarding vehicles 130 in detail,information regarding vehicles 131 arranged in the order according totheir priority levels with same-size images, and vehicle 130 having thehighest priority level whose image is emphasized by being colored.

(Operation)

Next, the following describes the operation of the informationprocessing method, information processing apparatus 150, and informationprocessing system 101 according to the present embodiment. Specifically,the following describes the operation of information processing system101 performed while vehicle 130 is moving.

FIG. 9 is a sequence diagram illustrating the operation of informationprocessing system 101 according to Embodiment 2.

As shown in FIG. 9, vehicle 130 obtains and transmits vehicle stateinformation to information processing apparatus 150 and terminal 170(step S201). Determination unit 152 of information processing apparatus150 receives the vehicle state information via communication unit 154.

In addition, vehicle 130 obtains vehicle position information indicatingthe current position of vehicle 130, and transmits the vehicle positioninformation to information processing apparatus 150 and terminal 170(step S202). Determination unit 152 of information processing apparatus150 receives the vehicle position information via communication unit154.

Vehicle 130 obtains sensor information indicating a passenger stateinside vehicle 130, vehicle's surroundings, that is, an object outsidevehicle 130, the position of the object, etc., and transmits theobtained sensor information to information processing apparatus 150 andterminal 170 (step S203). Determination unit 152 of informationprocessing apparatus 150 receives the sensor information viacommunication unit 154.

Detection processor 151 generates vehicle inside and outside informationindicating the passenger state inside vehicle 130, the position of theobject outside vehicle 130, etc., based on the sensor informationreceived in step S203 (step S204). Detection processor 151 outputs thevehicle inside and outside information to determination unit 152 (stepS205).

Determination unit 152 requests from storage controller 153 previousvehicle data similar to the obtained vehicle inside and outsideinformation (step S206).

In response to the request from determination unit 152, storagecontroller 153 searches for the previous vehicle data similar to thevehicle inside and outside information, and transmits the similarprevious vehicle data to determination unit 152 (step S207). Previousvehicle data is illustrated by FIG. 8A to FIG. 8C, and is informationincluding, for example, a location, the location corresponding to alabel, a recommended operation corresponding to the label, a dangerlevel corresponding to the location, the state of vehicle 130, and asituation corresponding to the location.

Determination unit 152 obtains the previous vehicle data from storagecontroller 153, and sets a priority level to each vehicle 130, formonitoring vehicles 130 on a priority basis (step S208). Determinationunit 152 sets the priority level based on a danger level. Thedetermination of the priority level will be described later.Determination unit 152 transmits information indicating the set prioritylevel of vehicle 130 to terminal 170 (step S209).

Next, terminal 170 receives the vehicle state information obtained instep S201, the vehicle inside and outside information, and theinformation indicating the priority level of vehicle 130 set in stepS209. Terminal 170 makes a presentation according to the priority levels(step S210).

Terminal 170 makes a presentation in various styles according to thepriority levels. FIG. 10A to FIG. 10F each illustrate a style of apresentation made by presentation unit 171.

FIG. 10A is a diagram illustrating a presentation made by terminal 170according to Embodiment 2. As shown in FIG. 10A, presentation unit 171presents vehicle 130 having the highest priority level with a large-areaimage. Specifically, presentation unit 171 presents only vehicles 130having at least a predetermined priority level among vehicles 130 andvehicle 130 having the highest priority level with a large-area image indetail. The information regarding vehicle 130 having the highestpriority level indicates the priority level, state, inside image,outside images, vehicle ID, current position, etc. of vehicle 130.Moreover, for example, presentation unit 171 presents, for vehicle 130having the second highest priority level or lower priority level,information simpler than that of vehicle 130 having the highest prioritylevel. Specifically, presentation unit 171 presents only one kind of animage regarding vehicle 130. Furthermore, presentation unit 171 does notpresent information regarding vehicle 130 having a priority level lowerthan the predetermined priority level, that is, vehicle 130 having thesixth highest priority level or a lower priority level in the presentembodiment.

FIG. 10B is a diagram illustrating a presentation made by terminal 170according to Embodiment 2. As shown in FIG. 10B, presentation unit 171presents only the information regarding vehicles 130 having at least apredetermined priority level among vehicles 130. In the presentembodiment, presentation unit 171 presents information regarding twovehicles 130 but not information regarding other vehicles 130 having thethird highest priority level or a lower priority level.

FIG. 10C is a diagram illustrating a presentation made by terminal 170according to Embodiment 2. As shown in FIG. 10C, presentation unit 171presents the information regarding vehicles 130 having the highestpriority level with the large-area images, and only vehicles 130 havingat least a predetermined priority level among vehicles 130. In FIG. 10C,presentation unit 171 presents every detail of the information regardingvehicles 130.

FIG. 10D is a diagram illustrating a presentation made by terminal 170according to Embodiment 2. As shown in FIG. 10D, presentation unit 171presents the information regarding vehicles 130 at a size that is largeenough for an operator to view the information regarding vehicles 130.In FIG. 10D, presentation unit 171 presents the information regardingvehicles 130 arranged in order of decreasing priority level.

FIG. 10E is a diagram illustrating a presentation made by terminal 170according to Embodiment 2. As shown in FIG. 10E, presentation unit 171presents the information regarding vehicle 3 having the highest prioritylevel that is emphasized by being colored.

FIG. 10F is a diagram illustrating a presentation made by terminal 170according to Embodiment 2. As shown in FIG. 10F, when vehicles 130 arecontinuously moving in column, presentation unit 171 collectivelypresents vehicles 130 as single vehicle 130.

It should be noted that the presentations in FIG. 10A to FIG. 10F aremere examples, and the present embodiment is not limited to these.

Here, we will return to the description of the sequence diagram shown inFIG. 9.

Terminal 170 receives an operating instruction from the operator (stepS211). For example, the operator checks on the operating status of eachvehicle 130 based on the presentation made by terminal 170. Then, theoperator inputs the operating instruction to terminal 170. When terminal170 receives the operating instruction from the operator, terminal 170transmits a control command to vehicle 130 presented by terminal 170(step S212). Here, the control command is a command for controlling themovement of vehicle 130, such as an emergency stop, a temporary stop,slow down, and steering operation of vehicle 130.

When vehicle 130 receives the control command transmitted in step S212,vehicle 130 performs an operation and display according to the controlcommand (step S213). In this manner, the processing by informationprocessing system 101 is ended.

Terminal 170 generates situation data indicating an evaluation accordingto the control command transmitted in step S212. For example, whendriving at reduced speed is performed due to an act of rushing out atpoint A, terminal 170 generates situation data indicating a location as“point A” a label as “danger of rushing out,” a recommended operation as“slow down,” and a danger level as “high.” It should be noted thatsituation data may be generated by the operator making an inputoperation via operating unit 172. In addition, terminal 170 mayautomatically generate situation data. Terminal 170 transmits thegenerated situation data to information processing apparatus 150 (stepS214). Storage controller 153 of information processing apparatus 150receives the situation data via communication unit 154. Storagecontroller 153 stores the situation data and updates a database instorage controller 153.

Moreover, determination unit 152 generates, for example, movementinformation indicating a position at which vehicle 130 has stopped,information about an object outside vehicle 130 at the time, etc., basedon the vehicle state information, vehicle position information, andsensor information, and outputs the movement information to storagecontroller 153 (step S215).

Storage controller 153 obtains the movement information and updates thedatabase in storage controller 153 (step S216). In this manner, theprocessing by information processing system 101 is ended.

It should be noted that, in information processing system 101, theupdate of the database in storage controller 153 from step S214 to stepS216 is not an essential process, and such a process need not beperformed.

The following describes how information processing apparatus 150 sets apriority level in detail. FIG. 11 is a flow chart illustratingdetermination of a priority level by the operation of informationprocessing apparatus 150 according to Embodiment 2. It should be notedthat a priority level may be determined using a trained model instead ofa database.

First, as shown in FIG. 11, information processing apparatus 150 obtainsvehicle state information, sensor information, and vehicle positioninformation (step S261).

Determination unit 152 of information processing apparatus 150determines whether vehicle 130 is abnormal based on the vehicle stateinformation (step S262). Specifically, determination unit 152 determineswhether the vehicle state information includes information indicating ahighly dangerous operation when vehicle 130 is moving, such as AEBactivation.

When vehicle 130 is determined to be abnormal (YES in step S262),determination unit 152 sets the priority level of vehicle 130 havingtransmitted the vehicle state information to be “high” (step S263).

Determination unit 152 transmits information indicating the prioritylevel of vehicle 130, which is set in step S263, to terminal 170 (stepS264).

When vehicle 130 is determined not to be abnormal (NO in step S262),determination unit 152 refers to a database regarding areas stored instorage controller 153 (step S265). Specifically, determination unit 152searches the database for a location (an area) in which vehicle 130 iscurrently moving, based on the vehicle position information.

Determination unit 152 determines whether the area in which vehicle 130is currently moving includes a danger point, based on the vehicleposition information (step S266). Examples of the danger point include apoint with danger of rushing out and a point under construction.

When the area in which vehicle 130 is currently moving is determined toinclude the danger point in the database (YES in step S266),determination unit 152 sets the priority level of vehicle 130, to whichthe information has been transmitted in step S261, to be “high” or“medium” (step S267), and proceeds to step S264.

When the area in which vehicle 130 is currently moving is determined notto include the danger point in the database (NO in step S266),determination unit 152 refers to a database regarding danger scenesstored in storage controller 153, based on the vehicle positioninformation and the sensor information (step S268). Specifically,determination unit 152 searches the database in storage controller forthe area in which vehicle 130 is currently moving, based on the vehicleposition information and the sensor information. Here, a danger scene isa scene in which vehicle 130 is expected to face a danger in an area inwhich vehicle 130 has difficulty self-driving. For example, a dangerscene is a case in which a bicycle is moving on the same road as vehicle130, a case in which a bicycle is approaching vehicle 130 from behind,and a case in which there are many blind spots due to on-street parkingand there is the danger of rushing out.

Determination unit 152 determines whether the area in which vehicle 130is currently moving includes the danger scene, based on the vehicleposition information and the sensor information (step S269).

When the area is determined to include the danger scene (YES in stepS269), determination unit 152 sets the priority level of vehicle 130, towhich the information has been transmitted in step S261, to be “high” or“medium” (step S270), and proceeds to step S264.

When the area is determined not to include the danger scene (NO in stepS269), determination unit 152 sets the priority level of vehicle 130, towhich the information has been transmitted in step S261, to be “low”(step S271), and proceeds to step S264. As described above, informationprocessing apparatus 150 determines the priority level and ends theprocessing. Although there are three priority levels of “high,”“medium,” and “low” in the present embodiment, there may be two prioritylevels or at least four priority levels.

It should be noted that a danger level is thought to differ depending onan area in which vehicle 130 is moving, the weather of the area, etc.Accordingly, for example, when vehicle 130 is moving on a narrow road orrain, fog, etc. has started, determination unit 152 may set the prioritylevel to be “high” in step S267 and step S270, the priority level to be“very high” in step S263, and the priority level to be “medium” in stepS267. Determination unit 152 may obtain weather information about thearea in which vehicle 130 is moving from an external server viacommunication unit 154.

As above, in the information processing method, information processingapparatus 150, and information processing system 101, it is possible tocalculate the danger level for each vehicle 130 based on the sensorinformation indicating the internal state and external state of vehicle130 when the operator monitors moving vehicles 130. Informationprocessing apparatus 150 presents, as vehicle 130 to be monitored on apriority basis, vehicle 130 having a higher outside vehicle dangerlevel, based on the danger level. Terminal 170 presents: vehicle 130having a high priority with the large-area image; only vehicle 130having the priority level higher than or equal to the threshold value;detailed information regarding all vehicles 130; information regardingvehicles 130 as much as possible; and vehicle 130 having the highestpriority level whose image is emphasized by being colored. For thisreason, the operator can monitor only vehicles 130 presented by terminal170, that is, more accurately find out vehicles 130 to be monitored. Asa result, it is possible to further assure safety for the vehicles bymonitoring the vehicles.

OTHER VARIATIONS

Although the information processing methods, information processingapparatuses 5 and 150, and information processing systems 1 and 101according to Embodiments 1 and 2 of the present disclosure have beendescribed above, the information processing methods, informationprocessing apparatuses 5 and 150, and information processing systems 1and 101 are not limited to aforementioned Embodiments 1 and 2.

For example, in the information processing method, informationprocessing apparatus 5, and information processing system 1 according toaforementioned Embodiment 1, the example is given in which the style ofthe presentation for monitoring or controlling vehicles 3 is determinedaccording to the boarding and alighting passengers, a style of apresentation may be determined according to the congestion degrees ofvehicles 3. For example, detection processor 51 detects the numbers ofpassengers, the numbers of empty seats, etc. inside vehicles 3 fromvehicle inside and outside images obtained by fourth obtaining unit 34,and generates congestion degree information. Determination unit 52determines priority levels based on the congestion degree information.Presentation controller 55 controls a style of a presentation based onthe priority levels.

Moreover, when the same priority level is determined for vehicles 3 and130 in the information processing methods, information processingapparatuses 5 and 150, and information processing systems 1 and 101according to aforementioned Embodiments 1 and 2, information processingapparatuses 5 and 150 may notify vehicles 3 and 130 about the samepriority level, and an announcement may be made in vehicles 3 and 130based on the notification.

Moreover, in the information processing methods, information processingapparatuses 5 and 150, and information processing systems 1 and 101according to aforementioned Embodiments 1 and 2, a portion of an imagepresented in a style based on priority levels may be emphasized, theportion relating to information used to determine the priority levels,and the image may be presented in a distinguishable manner.Specifically, the information used to determine the priority levels maybe a detected object. For example, any boarding and alighting passengerin an image may be highlighted in Embodiment 1. Further, any pedestrianor vehicle in an image may be highlighted in Embodiment 2. Examples of astyle of highlighting may include adding a color, a pattern, or a frameto an object, and flashing of an object.

Moreover, each processing component included in the informationprocessing methods, information processing apparatuses 5 and 150, andinformation processing systems 1 and 101 according to aforementionedEmbodiments 1 and 2 is implemented as an LSI (large-scale integratedcircuit) that is typically an integrated circuit. The integratedcircuits may be implemented in a single chip individually, or in asingle chip that includes some or all of them.

Moreover, the method of circuit integration is not limited to LSI.Integration may be realized with a specialized circuit or a generalpurpose processor. A field-programmable gate array (FPGA) for whichprogramming can be performed after an LSI is fabricated or areconfigurable processor capable of reconfiguring connections andsettings of circuit cells of an LSI may be used, instead.

It should be noted that each of the structural components in each ofaforementioned Embodiments 1 and 2 may be configured in the form of anexclusive hardware produce, or may be implemented by executing asoftware program suitable for the structural component. Each of thestructural components may be implemented by means of a program executingunit, such as a CPU or a processor, reading and executing the softwareprogram recorded on a recording medium such as a hard disk or asemiconductor memory.

Moreover, numbers in the above description are examples used forspecifically describing the present disclosure, and Embodiments 1 and 2of the present disclosure are not limited by such numbers.

Moreover, the block diagrams illustrate one example of the division offunctional blocks: a plurality of functional blocks may be implementedas a single functional block, a single functional block may be broken upinto a plurality of functional blocks, and part of one function may betransferred to another functional block. In addition, functions of aplurality of functional blocks having similar functions may be processedin parallel or by time-division by a single hardware or softwareproduct.

Furthermore, since the processing order of the steps in each flow chartis one example given for specifically describing the present disclosure,other processing orders may be adopted. In addition, a part of the stepsmay be performed simultaneously (in parallel) with another step.

The information processing methods, information processing apparatuses 5and 150, and information processing systems 1 and 101 according to oneor more aspects of the present disclosure have been described abovebased on Embodiments 1 and 2, Embodiments 1 and 2 of the presentdisclosure are not limited to the one or more aspects. Although onlysome exemplary embodiments of the present disclosure have been describedin detail above, those skilled in the art, will readily appreciate thatmany modifications are possible in the exemplary embodiments withoutmaterially departing from the novel teachings and advantages of thepresent disclosure. Accordingly, all such modifications are intended tobe included within the scope of the present disclosure.

It should be noted that, as described in Embodiments 1 and 2, machinelearning may be used for the processes performed by detection processors51 and 151 and determination units 52 and 152. Examples of machinelearning include: supervised learning that learns a relationship betweeninput and output using teaching data in which a label (outputinformation) is given to input information; unsupervised learning thatbuilds a data structure from only unlabeled inputs; semi-supervisedlearning that makes use of both labeled and unlabeled data; andreinforcement learning that learns continuous actions capable ofreceiving the greatest reward, by receiving feedback (reward) foractions selected based on state observation results. Further, examplesof a specific technique of machine learning include neural networks(including deep learning using multi-layer neural networks), geneticprogramming, decision trees, Bayesian networks, and support vectormachines (SVMs). The present disclosure may use any one of the abovespecific examples.

INDUSTRIAL APPLICABILITY

The present disclosure is applicable to devices that remotely operateself-driving vehicles, terminals that present the states of self-drivingvehicles, or systems including these devices. In particular, the presentdisclosure is applied to self-driving buses, self-driving taxis, andself-driving trains.

What is claimed is:
 1. An information processing method, comprising:obtaining, by an information processing apparatus, from vehicles, sensordata obtained from sensors of the vehicles; performing, by theinformation processing apparatus, object detection processes for eachvehicle using a different one of the sensor data; determining, by theinformation processing apparatus, a style of a presentation for anoperator to monitor or control the vehicles, using results of the objectdetection processes; and controlling the presentation made by apresentation device according to the style determined, wherein in eachof the object detection processes for each vehicle of the vehicles, anobject outside a moving vehicle and a position of the object aredetected, wherein the determining of the style of the presentationincludes: calculating, by the information processing apparatus, for eachvehicle of the vehicles, a danger level outside the vehicle, using theobject and the position of the object detected; determining, by theinformation processing apparatus, priority levels of the vehiclesaccording to the danger level outside the vehicle that has beencalculated for each vehicle so that the presentation is preferentiallymade to the operator; and determining the style of the presentation forthe operator according to the priority levels determined, the stylebeing a style in which a vehicle among the vehicles having a highercalculated danger level is presented as the vehicle to be preferentiallymonitored.
 2. An information processing apparatus, comprising: anobtaining unit that obtains, from vehicles, sensor data obtained fromsensors of the vehicles; a detection processor that performs objectdetection processes for each vehicle using a different one of the sensordata; a determination unit that determines a style of a presentation foran operator to monitor or control the vehicles, using results of theobject detection processes; and a presentation controller that controlsthe presentation made by a presentation device according to the styledetermined, wherein in each of the object detection processes for eachvehicle of the vehicles, an object outside a moving vehicle and aposition of the object are detected, and wherein the determination unitcalculates, for each vehicle of the vehicles, a danger level outside thevehicle, using the object and the position of the object detected,wherein the determination unit determines priority levels of thevehicles according to the danger level outside the vehicle that has beencalculated for each vehicle so that the presentation is preferentiallymade to the operator, and wherein the determination unit determines thestyle of the presentation for the operator according to the prioritylevels determined, the style being a style in which a vehicle among thevehicles having a higher calculated danger level is presented as thevehicle to be preferentially monitored.